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ABSTRACT 

The  digital  opacity  compliance  system  (DOCS)  has  been 
proposed  as  an  alternative  to  the  U.S.  Environmental  Pro¬ 
tection  Agency  Reference  Method  9  (Visual  Determina¬ 
tion  of  the  Opacity  of  Emissions  for  Stationary  Sources). 
The  DOCS,  which  employs  standard  digital  photography 
to  estimate  the  opacity  of  visible  emissions,  was  evaluated 
in  a  high  mountain  desert  environment  located  in  Weber 
County,  UT.  The  DOCS  recorded  an  average  opacity  de¬ 
viation  of  5.28%  when  applied  to  black  smoke  plumes 
having  true  opacities  in  the  range  of  0-100%,  an  error  rate 
that  was  found  to  be  significantly  less  than  7.5%  (allow¬ 
able  error  rate  for  attaining  certification  under  Method  9). 
In  contrast,  results  from  estimating  the  opacity  of  white 
smoke  plumes  indicated  that  the  accuracy  of  the  DOCS 
was  less  than  the  Method  9  error  rate  only  in  the  opacity 
range  of  0-60%,  over  which  the  DOCS  average  opacity 
deviation  was  determined  to  be  6.7%.  For  the  0-40% 
opacity  range,  the  DOCS  recorded  an  average  opacity 
deviation  of  5.44%  and  5.9%  for  black  and  white  plumes, 
respectively. 


IMPLICATIONS 

Historically,  Method  9  has  been  the  preferred  enforcement 
approach  for  verifying  compliance  with  federal  visible  opac¬ 
ity  standards  because  it  is  flexible  and  inexpensive  relative 
to  the  costs  associated  with  the  purchase  and  operation  of 
a  continuous  opacity  monitor.  The  DOCS,  which  is  an  in¬ 
novative  technology  that  employs  digital  photography  for 
quantifying  visible  opacity,  has  been  proposed  as  an  alter¬ 
native  to  Method  9.  The  DOCS  not  only  exhibits  compara¬ 
ble  performance  flexibility,  but  it  is  also  economically  com¬ 
petitive  and,  under  conditions  of  regulatory  concern  (i.e., 
0-40%  visible  opacity),  generates  opacity  measurements 
that  are  at  least  as  accurate  as  those  from  Method  9. 


Results  from  the  present  study  suggest  that  the  DOCS 
has  the  potential  to  quantify  visible  opacity  with  an  error 
rate  that  is  significantly  less  than  the  Method  9  permissi¬ 
ble  error  rate.  Although  encouraging,  it  is  unclear  to  what 
extent  the  DOCS  is  affected  by  climatic  conditions  other 
than  those  encountered  in  a  dry  desert  environment. 
Future  studies  should  focus  on  evaluating  the  perfor¬ 
mance  of  the  DOCS  under  variable  weather  conditions. 

INTRODUCTION 

The  presence  of  visible  air  emissions  from  industrial  op¬ 
erations  provides  irrefutable  evidence  that  airborne  parti¬ 
cles  are  being  discharged  into  the  atmosphere.  Not  only 
has  the  public  expressed  its  concern  regarding  the  nega¬ 
tive  psychological  effects  of  visible  emissions,  but,  under 
many  circumstances,  particulate  emissions  have  been 
identified  as  the  cause  of  increased  human  health  and 
environmental  risk.1  Because  of  public  concerns  regarding 
the  potential  health  and  environmental  impacts  associ¬ 
ated  with  visible  air  emissions,  current  federal  statutes  as 
well  as  many  state  and  local  air  quality  control  laws  cur¬ 
rently  regulate  the  opacity  of  plumes  (i.e.,  point  sources  of 
air  pollution)  as  well  as  fugitive  particulate  emissions. 
Opacity  is  defined  as  the  percentage  of  light  that  is  atten¬ 
uated  by  airborne  emissions.  In  other  words,  an  opacity 
value  of  20%  indicates  that  20%  of  the  incident  light  is 
absorbed,  scattered,  or  otherwise  blocked  by  the  plume 
while  80%  is  transmitted.2  In  many  cases,  the  U.S.  Envi¬ 
ronmental  Protection  Agency  (EPA)  and  state  regulatory 
authorities  employ  opacity  estimates  as  a  surrogate  mea¬ 
sure  to  verify  particulate  mass  emission  rate  compliance. 
Typically,  a  regulated  source  will  have  two  applicable 
particulate  matter  (PM)  limits,  one  that  specifies  a  maxi¬ 
mum  allowable  mass  emission  rate  (e.g.,  20  lb  of  PM/hr) 
and  another  that  specifies  a  maximum  acceptable  opacity 
(e.g.,  30%).  Although  the  goal  of  the  federal  opacity  limits 


724  Journal  of  the  Air  &  Waste  Management  Association 


Volume  53  June  2003 


McFarland  et  at. 


is  to  reduce  particulate  emissions,  EPA  reserves  the  right 
to  take  enforcement  action  against  any  regulated  source 
that  violates  its  opacity  standards  regardless  of  the 
source's  mass  emission  rate.  The  federal  opacity  standards 
for  emissions  from  various  industries  are  found  in  40  CFR 
Part  60  (Standards  of  Performance  for  New  and  Modified 
Stationary  Sources)  and  40  CFR  Parts  61  and  63  (Emission 
Standards  for  Hazardous  Air  Pollutants).3 

Although  EPA  requires  many  regulated  sources  to  in¬ 
stall  continuous  opacity  monitors  in  their  air  emission 
stacks,  the  primary  method  for  determining  compliance 
with  federal  opacity  standards  is  EPA  Reference  Method  9. 
Method  9  relies  on  the  ability  of  trained  human  observers 
to  visually  estimate  the  opacity  of  a  plume  by  taking  a 
series  of  opacity  measurements  at  the  rate  of  one  every  15 
sec  for  a  specified  period  of  time  (typically  6-60  min).2'4'5 
The  collective  set  of  opacity  measurements  is  then  aver¬ 
aged  to  develop  a  single  opacity  reading  for  the  regulated 
source,  which  is  subsequently  compared  against  permit¬ 
ted  levels.  To  qualify  as  a  Method  9 -certified  visual  ob¬ 
server,  an  individual  must  attend  and  successfully  com¬ 
plete  an  EPA-approved  Method  9  visual  opacity  "smoke 
school"  once  every  six  months.  Successful  completion  of 
an  EPA-approved  Method  9  smoke  school  requires  that 
the  candidate  enroll  in  both  classroom  and  field  training 
as  well  as  receive  a  passing  grade  on  an  opacity  field  test. 
To  achieve  a  passing  grade  on  the  opacity  field  test,  the 
candidate  must  demonstrate  the  ability  to  assign  an  opac¬ 
ity  reading  to  each  of  25  white  and  25  black  smoke 
plumes  with  a  margin  of  error  not  to  exceed  7.5%.6 

Advantages  claimed  for  employing  Method  9  to 
quantify  visible  opacity  have  included  the  fact  that  the 
approach  (1)  is  relatively  inexpensive,  that  is,  compared 
with  the  cost  of  purchasing  and  maintaining  a  transmis- 
someter  or  similar  device,  and  (2)  facilitates  enforcement 
by  allowing  a  regulator  to  take  compliance  measurements 
from  outside  the  facility's  property  line.  Although  the 
courts  have  generally  upheld  the  validity  of  Method  9  for 
compliance  demonstration  and  enforcement,  it  is  also  a 
well-known  fact  that  the  use  of  human  observers  to  quan¬ 
tify  visible  emissions  is  inherently  subjective,  a  character¬ 
istic  that  exposes  the  Method  9  results  to  charges  of  im¬ 
precision,  bias,  and  outright  fraud.  Moreover,  with 
increasing  public  pressure  to  reduce  airborne  PM,  regula¬ 
tors  will  continue  to  impose  greater  and  more  compre¬ 
hensive  limits  to  visible  emissions,  which  will  require, 
among  other  things,  a  commensurate  increase  in  both  the 
accuracy  and  reproducibility  of  the  methods  employed  to 
verify  the  level  of  visible  opacity.7 

The  current  field  study  was  designed  to  evaluate  the 
technical  performance  of  the  Digital  Opacity  Compliance 
System  (DOCS)  at  an  EPA-approved  smoke  school  con¬ 
ducted  in  a  high  mountain  desert  environment.  The  DOCS 


uses  a  commercial  off-the-shelf  (COTS)  digital  camera  to 
capture  images  of  visible  opacity,  which  are  then  down¬ 
loaded  to  a  standard  personal  computer  and  analyzed  using 
proprietary  commercially  available  software.8  Technological 
and  economic  advantages  claimed  by  the  developers  of  the 
DOCS  over  Method  9  have  included  (1)  minimization  of 
human  bias  associated  with  estimates  of  visible  opacity,  (2) 
creation  of  a  permanent  record  of  visible  emissions  for  ad¬ 
dressing  compliance  demonstration  requirements  and  pos¬ 
sible  legal  challenges,  and  (3)  reduction  in  the  annual  air 
compliance  costs  for  an  affected  facility. 

BACKGROUND 

In  the  DOCS,  digital  photographs  of  visible  emissions  are 
taken  from  valid  positions  according  to  Method  9  speci¬ 
fications.  Once  downloaded  to  a  computer  on  which  the 
DOCS  proprietary  software  has  been  installed,  the  digital 
images  can  be  evaluated  for  opacity.  The  initial  steps  in 
analyzing  the  digital  image  for  opacity  include  (1)  acti¬ 
vating  the  DOCS  proprietary  opacity  program,  (2)  elec¬ 
tronically  retrieving  those  digital  photographs  that  are  to 
be  evaluated,  and  (3)  using  the  DOCS  proprietary  software 
to  draw  an  analysis  box  (or  grid)  around  that  portion  of 
the  visible  emissions  that  will  be  analyzed  (Figure  1). 

After  selection  of  the  analysis  box,  the  DOCS  software 
utilizes  standard  statistical  data  analysis  techniques  for  image 
processing  to  differentiate  the  visible  emissions  from  the 
background.9  Once  the  visible  emissions  image  data  have 
been  effectively  separated  from  the  digital  background  in¬ 
formation,  the  software  is  capable  of  estimating  visible  opac¬ 
ity.  Under  normal  circumstances,  the  user  of  the  DOCS 
software  rarely  needs  to  understand  the  complex  mathemat¬ 
ical  relationships  associated  with  digital  image  processing 
techniques.  Rather,  the  DOCS  software  user  simply  draws 
the  analysis  box  around  the  area  of  the  visible  emissions  to 
be  analyzed  and  the  software  completes  the  opacity  analysis.8 

EXPERIMENTAL  METHODS 

To  document  the  ability  of  the  DOCS  to  accurately  quan¬ 
tify  visible  opacity  in  a  high  mountain  desert  climate, 
the  DOCS  was  evaluated  at  an  EPA-approved  Method  9 


Figure  1.  The  analysis  box  drawn  around  the  visible  emissions. 
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certification  smoke  school  conducted  in  Weber  County, 
UT.  The  field  tests  were  conducted  October  2-4,  2001. 
The  Weber  County  test  site  has  a  reported  elevation  of 
4790  ft  above  sea  level  with  geographic  coordinates  of 
approximately  41°  07'  N  (latitude)  and  111°  58'  W  (lon¬ 
gitude).  The  range  of  climatic  data  recorded  over  the  three 
days  of  the  DOCS  testing  included  the  following  param¬ 
eters:  (1)  mean  air  temperature,  (2)  average  wind  speed, 
(3)  maximum  wind  speed,  (4)  wind  direction,  (5)  sky 
conditions,  (6)  relative  humidity,  (7)  visibility,  (8)  baro¬ 
metric  pressure,  (9)  precipitation,  (10)  horizontal  sun  an¬ 
gle,  and  (11)  vertical  sun  angle.  Methods  used  to  estimate 
the  value  of  each  climatic  parameter  are  summarized  in 
Table  1. 

Field  Activities 

During  the  DOCS  evaluation,  four  COTS  digital  cameras 
(Kodak  DC290  or  Kodak  DC265)  were  employed  to  pho¬ 
tograph  visible  emissions  generated  during  the  EPA-ap- 
proved  Method  9  certification  field  test.  The  DOCS  com¬ 
mercial  software  was  installed  and  tested  on  each  of  the 
cameras  before  any  photographs  were  taken.  No  technical 
adjustments  or  physical  modifications  of  the  cameras 
were  necessary  to  operate  the  DOCS  camera  software. 

Each  camera  was  positioned  on  a  tripod  to  provide  a 
clear  view  of  the  visible  emissions.  The  minimum  dis¬ 
tance  of  the  cameras  from  the  stack  was  equivalent  to  at 
least  three  stack  heights  with  the  sun  oriented  in  the  140° 
sector  to  the  back  of  the  camera/observer.  Because  the 
emissions  stack  used  in  the  EPA-approved  smoke  school 
was  approximately  15  ft  high,  DOCS  cameras  and  visual 
observers  were  required  to  take  visual  opacity  measure¬ 
ments  at  a  distance  of  no  less  than  50  ft  from  the  smoke 


Table  1.  Methods  used  to  estimate  the  value  of  various  climatic  parameters. 


Parameter 

Method 

Mean  temperature 

Standard  liquid  thermometer  (fester  Technical  Associates, 
Inc.) 

Average  wind  speed 

Standard  anemometer  (fester  Technical  Associates,  Inc.) 

Max  wind  speed 

National  Weather  Service  (National  Oceanic  and  Atmospheric 
Administration) 

Wind  direction 

Standard  anemometer  (Easter  Technical  Associates,  Inc.) 

Sky  conditions 

Visual  observation 

Relative  humidity 

Sling  psychrometer  (Easter  Technical  Associates,  Inc.) 

Visibility 

National  Weather  Service  (National  Oceanic  and  Atmospheric 
Administration) 

Barometric  pressure 

National  Weather  Service  (National  Oceanic  and  Atmospheric 
Administration) 

Precipitation 

National  Weather  Service  (National  Oceanic  and  Atmospheric 
Administration) 

Horizontal  sun  angle 

Magnetic  compass  (fester  Technical  Associates,  Inc.) 

Vertical  sun  angle 

Abney  level  (fester  Technical  Associates,  Inc.) 

stack.  Two  cameras  were  placed  directly  in  line  between 
the  sun  and  the  smoke  stack  while  each  of  the  two  re¬ 
maining  cameras  was  placed  at  the  farthest  allowable 
azimuth  angle  (Figure  2).  Verification  of  the  horizontal 
sun  angle  was  established  using  a  standard  magnetic  com¬ 
pass  and  Abney  level  device  (Eastern  Technical  Associates, 
Inc.).  These  field  procedures  were  adopted  to  be  consis¬ 
tent  with  the  published  requirements  for  valid  Method  9 
visible  emissions  opacity  measurements.4'5 

Digital  photographs  taken  during  each  day  of  the  DOCS 
testing  were  collected  on  a  128-Mb  memory  card.  Following 
completion  of  each  day  of  testing,  the  used  memory  cards 
were  removed  from  the  cameras,  placed  in  a  labeled  con¬ 
tainer,  and  stored  in  a  secure  location.  Digital  photographs 
from  the  used  memory  cards  were  downloaded  daily  to  a 
laptop  computer  for  subsequent  opacity  measurement. 

In  addition  to  the  four  digital  cameras  used  during  the 
DOCS  evaluation,  an  EPA-certified  Method  9  smoke  reader 
(i.e.,  human  observer)  was  assigned  to  each  camera  location 
to  read  plume  opacities  during  the  first  day  of  DOCS  testing. 
The  objective  of  assigning  a  limited  number  of  EPA-certified 
Method  9  human  observers  to  estimate  plume  opacity  was 
to  develop  a  preliminary  comparison  of  the  relative  accuracy 
of  certified  human  observers  to  the  DOCS.  Unlike  the  EPA 
Method  9  certification  field-testing  procedures,  the  certified 
human  observers  were  not  provided  an  opportunity  to  cal¬ 
ibrate  their  vision  before  estimating  plume  opacity.5  The 
EPA-approved  smoke  generator  that  was  used  during  the 
subsequent  Method  9  certification  field  test  was  initially 
employed  in  the  one-day  certified  human  observer/DOCS 
side-by-side  evaluation. 

Opacity  Determination  Using  the  DOCS 

After  completion  of  field  data  collection  activities,  the 
opacity  of  each  smoke  plume  captured  as  a  digital  image 
at  the  Weber  County  smoke  school  was  estimated  using 
the  DOCS  proprietary  software  by  an  independent  eight- 
member  panel  that  consisted  of  federal  government  civil¬ 
ian  personnel,  U.S.  military  personnel,  and  federal  gov¬ 
ernment  contractors.  Each  panel  member  was  provided  a 
compact  disc  that  contained  all  the  digital  photographs 
taken  from  the  Weber  County  smoke  school  as  well  as  the 
DOCS  proprietary  software  and  user  guide.  The  panel  mem¬ 
bers  were  required  to  work  independently  to  estimate  the 
plume  opacity  of  each  digital  photograph  using  the  fur¬ 
nished  software.  Once  panel  members  had  completed 
their  analysis,  the  opacity  results  were  transferred  and 
stored  electronically  in  a  relational  database  for  subse¬ 
quent  statistical  evaluation.  An  independent  quality  con¬ 
trol  officer  was  assigned  the  responsibility  of  maintaining 
the  integrity  of  all  opacity  data  including  the  opacity 
results  generated  from  the  EPA-certified  transmissometer 
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Figure  2.  Positioning  of  the  digital  cameras  (Cl ,  C2,  C3,  and  C4)  and 
visual  observers  during  the  DOCS  field  study. 

against  which  the  DOCS  and  human  observer  opacity 
data  were  compared. 

Evaluation  of  the  DOCS  to  accurately  measure  visible 
opacity  was  based  on  comparing  the  opacity  measurements 
of  black  and  white  smoke  plumes  reported  by  the  eight- 
member  DOCS  panel  to  those  recorded  by  the  EPA-certified 
in-line  transmissometer.  Under  these  test  conditions,  those 
opacity  levels  recorded  by  the  transmissometer  were  as¬ 
sumed  to  reflect  the  "true"  (i.e.,  actual)  visible  opacity. 

Statistical  Evaluation  of  the  DOCS  Field  Data 

In  evaluating  the  DOCS  as  a  technically  defensible  alterna¬ 
tive  to  Method  9  for  estimating  visible  opacity,  the  absolute 
value  of  the  average  opacity  deviation  (i.e.,  differences  in 
opacity  readings  recorded  by  the  DOCS  and  the  EPA-ap- 
proved  transmissometer)  for  both  black  and  white  smoke 
plumes  was  computed  and  compared  with  the  Method  9 
error  rate  of  7.5%.2'5  Moreover,  to  ensure  that  technological 
decisions  resulting  from  the  field  data  could  be  supported 
with  a  known  degree  of  confidence,  the  uncertainty  associ¬ 
ated  with  the  opacity  deviation  measurement  was  quanti¬ 
fied  by  computing  its  99%  confidence  interval.  Equations 
1-4  were  employed  to  estimate  the  average  opacity  devia¬ 
tion  and  its  99%  confidence  interval. 

Average  opacity  deviation 


!  =  n 


Id, 

1=1 


l  i=n 

-I(YU-Y2,d 

i  =  1 


(1) 


I  (d,~d) 

Sample  variance  =  Sl~  — n  -  1 —  (2) 


Sd 

Standard  error  =  Sa  =  (3) 

99%  confidence  interval  =  d  ±  Sa  •  t(al 2),n  - 1  (4) 

where  dt  is  opacity  deviation  for  each  paired  observation, 
n  is  number  of  paired  observations,  Yu  is  the  ith  opacity 
level  as  measured  by  the  DOCS,  YZti  is  the  ith  opacity  level 
as  measured  by  the  EPA-certified  transmissometer,  a  is 
level  of  significance  (0.01  for  the  99%  confidence  inter¬ 
val),  n  -  1  is  degrees  of  freedom,  and  ta/2/  n_!  is  value  of  t 
statistic  obtained  from  standard  t-distribution  tables  (note 
for  n  -  1  >  120,  f0.oi/2,  n-i is  2.326). 

RESULTS 

Weather  conditions  during  the  3-day  test  period  were 
near  ideal  for  quantifying  visible  opacity.  The  climate  was 
basically  characterized  by  light  winds,  clear  blue  skies, 
and  no  precipitation.10  The  values  of  the  various  climatic 
parameters  monitored  during  the  field  test  period  are 
summarized  in  Table  2. 

During  the  Weber  County  smoke  school  test,  the 
DOCS  photograph  readers  performed  6928  opacity  esti¬ 
mates  from  digital  photographs  of  plumes.  Of  those,  only 
4767  were  deemed  valid  (2357  opacity  estimates  of  black 
plumes  and  2410  opacity  estimates  of  white  plumes).  The 
decision  to  exclude  2161  opacity  estimates  from  the 
DOCS  statistical  analyses  was  based  on  a  number  of  tech¬ 
nical  problems,  including  (1)  physical  obstruction  of  the 
smoke  plume  (e.g.,  trees,  clouds,  telephone  poles);  (2) 
folding,  twisting,  or  other  significant  physical  disruptions 
to  the  plume;  and  (3)  modification  of  digital  image 
through  use  of  the  "brush"  function  in  the  DOCS  soft¬ 
ware.8 

The  DOCS  "brush"  function  is  a  capability  the  DOCS 
developers  added  to  the  software  that  allows  the  technology 


Table  2.  Climatic  parameters  measured  or  reported  during  the  DOCS  field  tests. 


Day  of  Test 

Day  1 

Day  2 

Day  3 

Mean  temperature  (°F) 

66.2 

60.8 

60.8 

Average  wind  speed  (mph) 

8.9 

9.7 

8.1 

Max  wind  speed  (mph) 

13.8 

16.1 

11.4 

Wind  direction3 

NNW 

ESE 

SSE 

Sky  conditions 

Clear 

Clear 

Clear 

Relative  humidity  (%) 

27.2 

45.2 

30.5 

Visibility  (mi) 

7 

7 

7 

Barometric  pressure  (in.  Hg) 

30.08 

30.11 

30.01 

Precipitation  (in.) 

0 

0 

0 

Vertical  sun  angle  (°) 

42.2 

39.4 

38.5 

aNNW  =  north  by  northwest,  ESE  =  east  by  southeast,  SSE  =  south  by  southeast. 
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user  to  digitally  capture  a  part  of  the  background  of  a  valid 
photograph  and  utilize  it  to  enhance  the  contrast  of  a 
plume  (in  other  words,  the  software  user  can  effectively 
brush  in  the  background  based  on  a  particular  sky  condi¬ 
tion  found  within  the  photograph).  Although  the  DOCS 
photograph  readers  were  initially  allowed  to  utilize  the 
"brush"  function  if  they  determined  it  could  improve 
their  ability  to  evaluate  plume  opacity,  it  was  later  deter¬ 
mined  that  its  use  could  potentially  result  in  introducing 
bias  into  the  statistical  evaluation.  Therefore,  it  was  de¬ 
cided  that  all  DOCS  analyses  that  had  been  performed 
using  the  brush  function  would  be  deemed  invalid  and 
would  not  be  included  in  the  statistical  analyses. 

Quantitative  Analysis 

Table  3  summarizes  the  statistical  results  from  the  Weber 
County  smoke  school  DOCS  evaluation.  The  large  differ¬ 
ence  in  the  number  of  readings  recorded  by  the  DOCS 
relative  to  the  number  reported  by  certified  human  ob¬ 
servers  was  because  one  digital  photograph  could  be  eval¬ 
uated  by  as  many  as  eight  DOCS  photograph  readers  (i.e., 
number  of  members  on  the  digital  image  evaluation 
panel),  while  the  certified  human  observer  only  provided 
one  opacity  reading  per  smoke  plume. 

Over  the  full  range  of  opacity  (i.e.,  0-100%),  the 
average  opacity  deviation  of  the  DOCS  for  black  smoke 
was  estimated  to  be  6.58%  with  a  99%  confidence  interval 
that  ranged  from  6.1  to  7.  These  results  demonstrate  that, 
for  black  smoke,  the  DOCS  had  a  margin  of  error  in 
measuring  plume  opacity  that  was  significantly  less  than 
the  acceptable  error  rate  associated  with  Method  9  (i.e., 
7.5%).  Moreover,  the  range  of  the  99%  confidence  inter¬ 
val  indicates  that  there  was  less  than  a  1%  chance  of  the 
true  average  opacity  deviation  being  greater  than  7%. 
These  statistical  results  support  the  conclusion  that  the 


accuracy  of  the  DOCS  to  quantify  the  visible  opacity  of 
black  smoke  was  equal  to  or  greater  than  that  established 
for  the  Method  9  procedure.  Similarly,  in  evaluating  the 
ability  of  certified  smoke  readers  to  measure  the  opacity  of 
black  smoke  over  the  full  opacity  range,  the  average  opac¬ 
ity  deviation  was  estimated  to  be  7.45%.  Although  the 
average  deviation  was  below  the  acceptable  7.5%  error 
rate  associated  with  the  Method  9  procedure,  the  99% 
confidence  interval  included  deviations  that  were  greater 
than  7.5%,  a  fact  that  suggests  that  there  would  be  a  high 
degree  of  uncertainty  associated  with  the  conclusion  that 
the  certified  human  readers  could  reliably  maintain  the 
minimum  Method  9  accuracy  requirement. 

In  contrast  to  black  smoke,  the  field  data  indicated 
that  over  the  full  range  of  opacity  (0-100%),  neither  the 
DOCS  nor  the  certified  human  observers  could  measure 
the  opacity  of  white  plumes  with  the  minimum  Method  9 
accuracy.  Both  opacity  measurement  approaches  yielded 
average  opacity  deviations  that  were  significantly  greater 
than  the  acceptable  margin  of  error  associated  with 
Method  9.  The  failure  of  the  DOCS  to  accurately  measure 
white  plumes  over  the  full  range  of  opacity  stems  primar¬ 
ily  from  the  inability  of  the  proprietary  software  to  ex¬ 
clude  the  effects  of  shadows,  which  were  characteristic  of 
high-opacity  white  plumes  (i.e.,  those  having  opacities  of 
greater  than  60%). 

Despite  the  failure  of  the  DOCS  to  meet  the  mini¬ 
mum  Method  9  accuracy  requirements  during  opacity 
measurements  of  white  smoke  plumes,  when  the  statisti¬ 
cal  procedures  were  applied  to  a  limited  range  of  visible 
opacity,  the  DOCS  not  only  met  the  minimum  accuracy 
requirements  but  was  found  to  have  significantly  greater 
accuracy  than  Method  9 -certified  human  observers.  For 
example,  over  the  0-60%  opacity  range  for  white  smoke, 
the  average  opacity  deviation  of  DOCS  was  estimated  to 


Table  3.  Statistical  data  summary  of  Weber  County,  UT,  smoke  school  DOCS  evaluation. 


Color  of  Smoke 

Opacity  Measurement 
Approach 

Opacity  Range 
(*/<■) 

Average  Deviation 
|%) 

Number  of 
Samples 

99%  Cl* 

Black 

DOCS 

0-100 

6.58 

2357 

6.1-7 

Black 

Certified  observers 

0-100 

7.45 

280 

6-8.8 

Black 

DOCS 

0-60 

5.70 

1972 

5.3-6. 1 

Black 

Certified  observers 

0-60 

5.82 

225 

4. 5-7.1 

Black 

DOCS 

0-40 

5.44 

1759 

5-5.9 

Black 

Certified  observers 

0-40 

4.77 

194 

3.5-6. 1 

White 

DOCS 

0-100 

10.08 

2410 

9.5-10.7 

White 

Certified  observers 

0-100 

8.55 

282 

7.1-10 

White 

DOCS 

0-60 

6.70 

1900 

6.2-7. 2 

White 

Certified  observers 

0-60 

8.17 

224 

6.6-9. 8 

White 

DOCS 

0-40 

5.90 

1689 

S.4-6.4 

White 

Certified  observers 

0-40 

7.39 

199 

5.7-9 

aC!  =  confidence  interval. 
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be  6.7%  with  a  99%  confidence  interval  that  ranged  from 
6.2  to  7.2%.  These  data  indicate  that,  if  the  opacity  of  the 
white  plume  were  equal  to  or  less  than  60%,  the  DOCS 
accuracy  would  exceed  the  minimum  level  required  for 
demonstrating  equivalency  to  Method  9.  Furthermore, 
the  99%  confidence  interval  indicates  that  there  was  less 
than  a  1%  chance  of  the  true  average  deviation  being 
greater  than  7.2.  These  statistical  results  support  the  con¬ 
clusion  that  the  accuracy  of  the  DOCS  in  measuring  the 
opacity  of  white  smoke  over  the  range  of  0-60%  opacity 
was  significantly  greater  than  that  established  for  Method 
9.  In  contrast  to  the  ability  of  the  DOCS  to  attain  the 
Method  9  accuracy  requirements  when  its  application  was 
limited  to  the  0-60%  opacity  range  for  white  plumes, 
opacity  results  reported  by  the  certified  human  observers 
indicated  that  the  human  observers  exceeded  the  allowed 
Method  9  error  rate.  For  example,  over  the  limited  opacity 
range  of  0-60%,  the  average  opacity  deviation  associated 
with  certified  smoke  readers  was  estimated  to  be  8.17% 
with  a  99%  confidence  interval  that  ranged  from  6.6  to 
9.8%.  Finally,  it  should  be  noted  that,  over  the  range  of 
visible  opacity  levels  typically  established  for  regulated  air 
sources  (i.e.,  0-40%  opacity),  the  accuracy  of  the  DOCS 
was  significantly  greater  than  the  level  required  to  dem¬ 
onstrate  equivalency  to  Method  9,  regardless  of  the  color 
of  the  plume  (see  Table  3). 

To  more  fully  characterize  the  technical  limitations 
of  the  DOCS  in  quantifying  visible  emissions,  the  effect  of 
camera  position  on  opacity  measurement  was  evaluated. 
As  illustrated  in  Figure  2,  two  of  the  DOCS  cameras  were 
placed  directly  in  line  between  the  sun  and  the  smoke 
stack  (designated  as  camera  positions  C2  and  C3),  while 
the  two  remaining  DOCS  cameras  were  placed  at  the 
extreme  azimuth  angle  permissible  for  taking  valid 
Method  9  opacity  measurements  (identified  as  camera 


positions  Cl  and  C4).  Table  4  summarizes  the  statistical 
results  obtained  from  the  DOCS  and  certified  human  ob¬ 
server  comparison,  which  was  evaluated  at  the  various 
digital  camera/observer  positions. 

Comparison  of  the  opacity  deviations  as  a  function  of 
the  camera/human  observer  position  indicated  that,  as 
long  as  the  visible  emissions  were  observed  from  a  dis¬ 
tance  of  at  least  three  stack  heights  from  the  emission 
source  with  the  sun  contained  within  the  allowable  140° 
sector  as  prescribed  by  the  Method  9  protocol,  camera 
position  did  not  impact  the  accuracy  of  the  DOCS  in 
quantifying  opacity.  For  example,  in  estimating  the  visi¬ 
ble  emissions  associated  with  black  smoke,  the  most  ac¬ 
curate  readings  for  the  DOCS  were  taken  from  the  Cl 
position  (e.g.,  average  opacity  deviation  5.4%),  while  for 
the  certified  human  reader,  the  most  accurate  readings 
were  recorded  from  positions  C2  and  C3  (average  opacity 
deviation  6.38%).  Conversely,  for  white  smoke,  the  most 
accurate  readings  for  both  the  DOCS  and  the  certified 
human  readers  were  from  Cl,  where  the  average  devia¬ 
tions  were  6.28  and  8.77%,  respectively.  These  data  sug¬ 
gested  that  the  specific  positioning  criteria  prescribed  for 
certified  human  observers  when  taking  valid  Method  9 
opacity  readings  could  be  adopted  by  the  DOCS  field 
protocol  without  any  adverse  effects  on  the  accuracy  of 
the  opacity  measurement. 

CONCLUSIONS 

The  field  results  obtained  in  the  present  study  clearly 
demonstrate  that  the  DOCS  has  the  potential  of  satisfying 
the  technical  and  quality  performance  standards  to  qual¬ 
ify  as  an  alternative  to  Method  9.  The  statistical  accuracy 
of  the  DOCS  in  quantifying  the  opacity  of  black  plumes 
was  found  to  be  significantly  greater  than  the  minimum 
level  established  for  measuring  visible  opacity  using 


Table  4.  Effect  of  digital  camera  position  on  the  accuracy  of  the  DOCS  measurement,3 


Color  of  Smoke 

Opacity  Measurement 
Approach 

Camera 

Position 

Opacity  Range 
(%) 

Average  Deviation 
(%) 

Number  of 
Samples 

99%  Cl6 

Black 

DOCS 

Cl 

0-100 

5.40 

603 

4.7-6. 1 

Black 

DOCS 

C2.C3 

0-100 

6.71 

939 

5.9-7.6 

Black 

DOCS 

C4 

0-100 

7.31 

815 

6.6-8. 1 

Black 

Certified  observers 

Cl 

0-100 

9.06 

106 

6.4-11.7 

Black 

Certified  observers 

C2,  C3 

0-100 

6.38 

149 

4.7-8. 1 

Black 

Certified  observers 

C4 

0-100 

7 

25 

3.4-10.6 

White 

DOCS 

Cl 

0-100 

8.77 

545 

7.5-10 

White 

DOCS 

C2,C3 

0-100 

10.63 

1176 

9.7-11.6 

White 

DOCS 

C4 

0-100 

10.20 

689 

9.1-11.3 

White 

Certified  observers 

Cl 

0-100 

6.28 

98 

4.6-7.9 

White 

Certified  observers 

C2,  C3 

0-100 

7.28 

112 

5.6-9 

White 

Certified  observers 

C4 

0-100 

13.61 

72 

9.7-17.5 

aSee  Figure  2  for  exact  camera/observer  location;  bCI  =  confidence  interval. 
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Method  9.  For  white  smoke;  the  accuracy  of  the  DOCS  is 
statistically  less  than  the  Method  9  error  rate  only  when 
the  actual  (i.e.;  true)  opacity  was  limited  to  the  range  of 
0-60%.  The  technical  reason  cited  by  the  developers  of 
the  DOCS  for  its  inability  to  accurately  quantify  the  opac¬ 
ity  of  dense  white  plumes  is  the  effect  of  shadows;  which 
are  often  associated  with  these  types  of  emissions.  An 
extensive  presence  of  shadows  within  the  digital  image 
decreases  the  ability  of  the  DOCS  proprietary  software  to 
effectively  differentiate  between  visible  emissions  and 
background;  which  can  result  in  significantly  reducing 
the  accuracy  of  the  opacity  measurement. 

From  a  regulatory  standpoint,  the  inability  of  the 
DOCS  to  accurately  measure  the  opacity  of  dense  white 
plumes  means  that  the  technology,  in  its  present  design, 
is  incapable  of  successfully  passing  an  EPA-approved 
smoke  school.  Despite  this  limitation,  for  those  opacity 
ranges  typically  included  in  regulatory  permits  (0-40%), 
the  accuracy  of  the  DOCS  was  found  to  exceed  the  min¬ 
imum  requirements  established  for  quantifying  opacity 
using  Method  9  regardless  of  plume  color.  The  effect  of 
camera  position  on  the  DOCS  accuracy  in  measuring 
plume  opacity  was  found  to  be  negligible,  provided  that 
the  digital  photographs  were  taken  from  locations  that 
were  consistent  with  the  human  observer  positioning  cri¬ 
teria  specified  by  Method  9.  Moreover,  like  Method  9,  the 
DOCS  cannot  reliably  quantify  opacity  levels  of  visible 
emissions  when  the  plume  is  subject  to  folding  or  twisting 
or  is  obstructed  by  physical  objects. 

Finally,  in  developing  the  scope  of  the  DOCS  exper¬ 
imental  field  design,  it  was  recognized  that,  because  of  its 
dependency  on  electronically  powered  optical  equip¬ 
ment,  the  response  of  the  DOCS  could  vary  significantly 
depending  on  climatic  conditions.  In  other  words,  tem¬ 
perature,  humidity,  and  other  environmental  factors 
could  potentially  affect  the  accuracy  of  the  DOCS  in 
quantifying  visible  opacity.  Therefore,  the  present  field 
results  only  reflect  the  performance  of  the  DOCS  under 
low  humidity  and  high  visibility  climatic  conditions,  that 
is,  those  climates  typically  encountered  in  high  mountain 
deserts.  It  is  anticipated  that  analysis  of  the  DOCS  perfor¬ 
mance  at  EPA-approved  smoke  schools  conducted  in  hu¬ 
mid  climates  or  in  locations  with  a  high  degree  of  cloud 
overcast  will  provide  important  field  results  that  are  nec¬ 
essary  to  more  fully  describe  the  impact  of  climatic  con¬ 
ditions  on  the  technology's  ability  to  quantify  visible 
emissions. 
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